Data Munging

Data Munging
- Missing values
- Data Enrichments
- Data Melting
- Clean unicode
- Feature Engineering and Variables Transformations
Model builtding
- Data encoding
- Feature scaling
- Feature Selections

Missing Values

imputing https://github.com/rhiever/datacleaner

Variable Transformations and Feature engineering

text transfomations https://www.analyticsvidhya.com/blog/2014/11/text-data-cleaning-steps-python/
add sentiment

email validation http://stackoverflow.com/questions/8022530/python-check-for-valid-email-address
parsing usa address into its components using https://github.com/datamade/usaddress
parsing and phone numbers using https://github.com/daviddrysdale/python-phonenumbers
parsing names into name components using https://github.com/datamade/probablepeople
parsing user agent into its components using https://github.com/selwin/python-user-agents
parsing url into its url components using https://github.com/unshiftio/url-parse

Regular expressions: misspellings, regular expression patterns e https://www.analyticsvidhya.com/blog/2016/01/guide-data-exploration/

Data enrichment

parsing name into gender using https://github.com/ferhatelmas/sexmachine
parsing ip into location components using https://github.com/maxmind/geoip-api-python



In [ ]:

    
https://github.com/NathanEpstein/Dora
    Feature Selection & Extraction



In [ ]:



In [ ]:

    
https://github.com/LuminosoInsight/python-ftfy
        Given Unicode text, make its representation consistent and possibly less broken.



In [ ]:



In [ ]:

    
http://nbviewer.jupyter.org/github/rasbt/pattern_classification/blob/master/preprocessing/about_standardization_normalization.ipynb
About Feature Scaling: Z-Score Standardization and Min-Max Scaling



In [ ]:



In [ ]:

    
https://github.com/NathanEpstein/Dora
data versioning